1. Field of the Invention
The present invention relates to a lane recognition image processing apparatus which is installed on a vehicle for recognizing a lane of a road based on the sensed or picked-up image of lane markings on the road on which the vehicle is travelling, and which is applied to an advanced vehicle control system such as a lane departure warning system (LDWS) intended for use with preventive safety of the vehicle such as an automobile, a lane keeping system (LKS) serving the purpose of reducing a cognitive load on drivers, etc. More particularly, the invention relates to a technique capable of improving reliability in the result of the recognition by providing vehicle lateral or transverse position information in the lane.
2. Description of the Related Art
As a conventional lane recognition image processing apparatus, there has been known one using an image filter (for example, see a first patent document: Japanese patent application laid-open No. H10-320549 (JP, H10-320549, A)).
This type of image filter is constructed of a relatively simple circuit that can extract an area of a gray scale picture or image which is brighter than its surroundings and which is less than or equal to a predetermined width.
The processing disclosed in the above-mentioned first patent document is called a one-dimensional image filtering process in which the gray value g(h) of a pixel of interest is compared with the gray values g (h−Δh) and g (h+Δh) of pixels distant a kernel size Δh from the pixel of interest forwardly and rearwardly in a search scanning direction, and the smaller value of the differences {g (h)−g (h−Δh)} and {g (h)−g (h+Δh)} thus obtained is made to be a filter output value.
In the conventional lane recognition image processing apparatus, when the forward view of the vehicle is taken by a camera installed thereon in a direction in which the vehicle is travelling, objects on an image thus taken become linearly smaller toward a vanishing point. Therefore, when the width of the neighborhood of the pixel of interest (i.e., a kernel size Δh) to be referenced or viewed by a one-dimensional image filter is fixed, the actual width of an area extracted by the filter increases linearly in accordance with the increasing distance thereof from the camera. Accordingly, in case where a lane marking of a predetermined width on a road is detected as a physical quantity, the possibility of the presence of objects other than the lane marking becomes higher as the distance from the camera increases, so there arises a problem that reliability in the result of the recognition of a distant portion of the lane marking, which is needed to exactly grasp the shape of the road, is reduced.
In addition, when the image filter for use with the extraction of lane markings is applied to an road image that includes noise components of high intensity in a range on the road, there will be another possibility of misdetecting noise portions as lane markings. In particular, in case where a binarization threshold is controlled to decrease so as to extract degraded or thinned lane markings, or where a search area includes only high-intensity noise components but no lane marking such as in the case of discontinuous portions of an intermittent lane marking, there will be a problem that noise can be misdetected with a very high possibility.
Moreover, in the case of using a CMOS image sensor as an image sensing means, the CMOS image sensor is superior to a CCD image sensor with respect to the reduction in size and cost of peripheral circuits, but has a lower S/N ratio, so there is a higher possibility that the images taken by the CMOS image sensor contain noise. Accordingly, when the binarization threshold of the image filter is controlled as usual with respect to the images taken by the CMOS image sensor, the noise component passes through the filter, thus giving rise to a problem of decreasing lane marking recognition performance
Further, in recent years, CMOS image sensors with a wide dynamic range are being developed, and intermittent high intensity parts are becoming visually recognizable. However, when an image made to have a wide dynamic range is expressed as a gray scale image of a plurality of gradations (for instance, 256 steps), the entire image becomes a low contrast, so there arises a problem that in the ordinary control of the binarization threshold, it is often difficult to extract lane markings.
Furthermore, in the conventional lane recognition image processing apparatus, lane markings are extracted by using one binarization threshold with respect to one image. Thus, in general, the contrast in the output result of the image filter is high in near regions and low in distance regions, so there is a problem that in the case of extracting lane markings by the use of a single binarization threshold, it is impossible to extract a distance lane marking though a near lane marking can be extracted.
In addition, even if the binarization threshold is simply controlled to decrease in accordance with the increasing distance, there will happen a situation where the contrast can be varied at a distant or near location due to the shades of road structures depending upon the road-surrounding environment.
Moreover, in setting a window, in order to set the position of the window at a location including a lane marking and properly limit the size of the window, it is appropriate to set a current window based on the last window position calculated from a lane marking mathematical model equation, but in a situation where the number of extracted candidate points is limited and a lane marking mathematical model equation cannot be derived (i.e., the state of lane markings being lost sight of), there exists no setting reference position, so it is necessary to set a window of a wide or large size so as to search for a lane marking from the entire screen. At this time, an extended period of time for processing is required due to a wide or large search area. Therefore, it takes time for the condition to return from a lane marking lost-sight state to a lane marking recognition state, thus posing a problem that the performance of the lane recognition image processing apparatus is reduced to a substantial extent.
Further, in lane recognition image processing, it has been proposed to extract top-hat shapes (i.e., having a constant width and a luminance higher than that of the road surface) by using a one-dimensional image filter. However, such a proposal has a problem in that with respect to images of low contrast or images of low S/N ratios taken by an image sensor of a wide dynamic range, there is a possibility of misdetecting objects other than lane markings, and that once a lane marking is lost sight of, it takes time until recognition of the lane marking is restored.